This section provides context for various embodiments of the invention as recited in the claims. While the content may comprise subject matter that could be used, it has not necessarily been previously used or described. The content described in this section is not considered prior art unless otherwise indicated and should not be considered as admitted prior art due to inclusion in this section.
In general, existing data management and message processing systems do not provide the level of security required to sufficiently protect user information. Typical methods of protecting data utilize encryption of single columns, single data cells or encryption of storage, known as whole disk encryption. These methods require that the encryption keys be present or accessible to the data management server. These keys must be stored on the data management server or brought to the server for any operation requiring decryption of data within the system. This includes almost every useful data operation. Having the encryption keys available on the same servers as the encrypted data is a serious vulnerability. An attacker or insider that gains access to the servers or storage could get the keys and decrypt the encrypted data. Not only are the servers themselves vulnerable, but everywhere that the information is kept is vulnerable. This includes locations such as the server memory, on local disks, in network storage, in backup files, server replicas, and over the network. There are too many known and unknown vulnerabilities to these and other information technology (IT) components to be confident that the data is protected. Therefore, all of these data management system components require extraordinary security measures to ensure that nobody can steal the valuable information. Given the numerous vulnerabilities present in modern operating systems and software, it is incredibly challenging and expensive to secure all of the places that valuable information is kept.
FIG. 1 provides an example of a prior art data management system 100 that co-locates encryption keys with encrypted data. In this partially encrypted, yet insecure, data management system 100, client 101 sends operations and queries to server 102, which includes data engine 103, data indexes 104, storage 106, and backup files 107. The server 102 organizes information by row or column or key/value. The data is protected by encrypting each row, cell, column, or storage. Data indexes 104 are used by data management system 100 to quickly search for information. Data indexes 104 are typically some type of Btree. These data indexes 104 are not encrypted because they cannot be efficiently searched if they are encrypted. Data protection methods such as row, column, and whole storage encryption require that the encryption keys 105 be present on the server 102. Those keys 105 must be either stored on the server 102 or brought to the server 102 for any operation requiring decryption of data within the server 102. Storing encryption keys 105 on the same physical machines as the encrypted data creates a significant vulnerability within the system 100 as any attacker or insider that gains access to the server 102 could find the keys 105, decrypt the data with those keys 105, and steal the information present. For example, since database management system administrators have access to both the encryption keys 105 and the encrypted data, these insiders can steal the encrypted data and decrypt it. Additionally, if the data management system 100 is vulnerable to attackers, the encryption keys 105 may be stolen. Therefore, it is not safe to co-locate encrypted data with their encryption keys 105. However, no encryption system is fast enough to allow for operating on encrypted data without decrypting it. No alternatives have been found that are secure enough and operate quickly enough. A high performance and secure solution to this problem would provide substantial improvements to information security.